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(54) Blending colors in the presence of transparency 

(57) Methods and apparatus for blending colors in B)*a*(b/c). 
the presence of transparency. The method includes 
blending a lower and an upper graphic element in 
accordance with a transfer mode by calculating a trans- 
fer color from a lower element color and an upper ele- 
ment color according to the transfer mode without 
regard either to a lower element alpha or to an upper 
element alpha; calculating a resulting alpha from the 
lower element alpha and the upper element alpha with- 
out regard to any of the lower element color, the upper 
element color, or the transfer mode; and calculating a 
resulting color as a weighted average of the lower ele- 
ment color, the transfer color, and the upper element 
color. The transfer mode can be an arbitrary transfer 
mode and can be user defined. The resulting graphic 
element <C,c) having color value C and alpha value c 
can be calculated in accordance with 



(C,c) = XFade ( <A,a>, XFade ( <B,1>, <T(A,B), 1), a), 
b), 

where XFade «X,x), (Y,y), z) = (R,r) is an operation 
defined by the formulas 

CM r = InterpolateAlpha (x, y, z), 

^ t = y • z / r, and 

CM R = InterpolateColor (X, Y, t) 



CO 

o 
to 



a. 

LU 



and where the lower element is <A,a) and the upper ele- 
ment is (B,b). Each of the colon A, B, and C can be 
defined by a value for each of a plurality of colorants. 
The calculated values can defined by c = a + b - a • b, 
and C = A-(1 - (b / c)) + B • (1 - 1)*(b / c) + T(A, 
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Description 

BACKGROUND OF THE INVENTION 

5 [0001] The present invention is related to the compositing or blending of graphical image data. 

[0002] Many graphics or image processing computer programs, such as Adobe® Photoshop®, available from 
Adobe Systems Incorporated, of San Jose, California, build a final image by compositing two or more image layers 
together. The image layers may be thought of as stacked sheets of acetate. The density of the ink on the acetate con- 
trols the transparency of the sheet, i.e., the extent to which that sheet obscures the images from the underlying sheets. 

10 In the computer program, the color and density of the ink on the acetate sheet are represented by a color value and an 
alpha value (representing opacity), respectively. Normally, an alpha value of zero means the corresponding region is 
fully transparent, and a value of one means it is fully opaque. 

[0003] Each layer typically includes image data and compositing controls. Typically, the image data is represented 
by an array of pixels, with each pixel having a color and, optionally, an opacity. Similarly, if there is a mask, the mask is 
15 represented by an array of pixels, with each pixel having an opacity. Alternatively, the image data or the mask or both 
can be defined analytically, e.g., by using shape outlines, or by other functions which map positions to color and opacity. 
In addition, the image data and the mask can be dynamic, that is, computed from other data at the time the layers are 
composited. 

[0004] The compositing controls may include a transfer mode, also known as a blending mode. The transfer mode 
20 of an image layer determines how the color in the image layer mixes with the color accumulated from the underlying 
layers in the same image position. More specifically, it is a recipe for blending colors that can be expressed as a function 
taking an upper color and lower color to produce a third color. Image manipulation computer programs, such as Pho- 
toshop®, generally provide a wide variety of predefined transfer modes. The basic transfer mode equation is: 

25 new_color = a • T(lower_color, upper_color) + (1 -a) • lower_color 

where a is the opacity of the upper layer, T is the selected transfer mode function, and lower_color and upper_color are 
the color values of the lower layer (which in some contexts is called the accumulation layer) and the upper layer for the 
graphical element whose color new_color is being calculated. The color values are not pre-multiplied by the opacity. 

30 Transfer modes should not be confused with the well-known Porter-Duff compositing operations, which define how the 
alpha values determine how much of the blended colors survive in the result, but which assume essentially normal 
mode color mixing. T. Porter and T. Duff, "Compositing Digital Images", SIGGRAPH 84, pp. 253-59 (1984). 
[0005] The following table shows examples of transfer mode functions T. Each of the example transfer modes works 
componentwise; that is, each new colorant value is calculated based solely on the corresponding argument colorant 

35 values and independently of the values of different colorants. However, in the general case, transfer mode functions are 
not so limited. For example, there are hue, color, saturation, and luminosity modes that are based on doing a conversion 
to a hue, saturation, and luminance form (not exactly the same as either HSL or HSB because luminance is computed 
as a weighted average of red, green, and blue or cyan, magenta, and yellow multiplied by black), taking an appropriate 
mixture of the components from the upper and lower colors, and then performing a hue preserving clip to convert the 

40 color back into the RGB or CMY form. As is conventional, component colorants are treated as ranging from zero to one, 
with zero representing the darker end of the range, although in a typical implementation the actual values representing 
zero to one generally range across the values available at the colorants' bit depth. The exact effect of the blending mode 
will depend on the color space. The convention that zero is darker than one for each colorant means that 100% ink in 
CMYK is represented by zero and 0% ink is represented by one, whereas in RGB, no phosphor excitation is repre- 

45 sented by zero and full excitation is represented by one. 



T(L, U) 


Formula 


Normal (L, U) 
Multiply (L, U) 
Screen (L, U) 
Difference (L, U) 
ColorDodge (L, U) 
ColorBurn (L, U) 


= U 
= L*U 

= U + L-U • L=1 -(1 -U) • (1 -L) 
= IU - LI 

= Min (1 , L7(1 - U)), with 0/0 treated as 0 
= 1 - Min (1 ,(1 - L)/U), with 0/0 treated as 0 
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(continued) 



T(U U) 


Formula 


r> „ i /i i i\ 

Darker (L, U) 


= Min (L, U) 


1 ! — 1— J. — /I 1 1 \ 

Lighter (L, U) 


ft A / 1 1 1 \ 

= Max (L, U) 


Add (L, U) 


= Max (1 , L + U) 


Overprint (L, U) 


-i A _l _l / -4 1 -4 1 1 \ 

= 1 - Add (1 - L, 1 - U) 


q n ft| inht (\ I \\ 
oui Li_iy 1 1 L KJ J 


— L ^l t'UJ'L'^l LJII U.O 




= L + (2 • U - 1 ) • (D(L) - L) if U > 0.5, 




where 




D(x)=((16«x- 12)«x + 4)«xif x<0.25 




= sqrt (x) if x > 0.25 



[0006] Image layers are generally composited in order from bottom to top. The general process for compositing an 
image layer begins with calculation of any dynamic data in the image layer, such as the color of the pixels in the image 
20 and the opacity of the pixels in the mask. Then, the opacity is determined for each pixel from the mask, the global opac- 
ity, if any, and the image data. Finally, the color of each pixel in the layer is combined with the color of the corresponding 
pixel in an accumulation buffer to generate a new composited color. The combination is controlled by the opacity of the 
pixel and the transfer mode. 

[0007] Some computer programs, in addition to having predefined transfer modes, also give users the ability to 

25 define their own transfer modes. One such system is described in commonly-owned United States patent application 
number 09/008,269 filed January 16, 1998 to Hamburg, incorporated here by this reference, which discloses methods 
and apparatus, including computer programs, for use by a user to create new transfer modes (called compound transfer 
modes) based on existing (or newly-created) transfer modes and a scaler parameter, as appropriate. 
[0008] In this specification, the terms color, alpha, and graphic element are used. A color is a representation of a 

30 particular color. The representation can be in any form suitable for computation and need only have the property that 
colors can be interpolated. Alpha is a quantity that characterizes the opacity of a graphic element. An alpha value of 
zero indicates total transparency. An alpha value of one indicates complete opacity. When working with graphic ele- 
ments described with a color and alpha value, the color is free to be undefined or arbitrary if the alpha value is zero. A 
graphic element is a piece of the compositing process. Two graphic elements each having a color and alpha value are 

35 composited to produce a third graphic element. In a raster context, the primitive graphic element is a pixel. In a vector 
graphics context, it could be a region of solid color, such as a region defined by a Postscript path. 
[0009] Raster accumulation is one example of compositing. In raster accumulation, the primitive graphic element is 
the pixel. The accumulation buffer is a collection of pixels arranged in a regular pattern — generally a grid. The elements 
to be accumulated are also collections of pixels. Every pixel has a color and may have an alpha value. To composite a 

40 single element into the accumulation buffer, each pixel in the accumulation buffer is composited with the corresponding 
pixel in the element to produce a new value for the accumulation buffer pixel. A stack of elements is composited by ini- 
tializing the accumulation buffer and then compositing each element from bottom to top into the accumulation buffer. 
The final contents of the accumulation buffer represent the final image. 

[0010] At least one prior art system provided compositing with transfer mode T for color blending without alpha in 
45 an accumulation buffer (i.e., with an implicit alpha value of one) as follows. 

new__accumulation_color «= 

InterpolateColor {©ld_accumulation_color, 
so T {old_accuraulacionj:olor, element_color) , 

elemenr^alpha) 

55 where 

InterpolateColor (A, B, 0) = A 
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and 

InterpolateColor (A, B, 1) = B. 

5 Conventionally, color interpolation is linear, and defined as 

InterpolateColor (A, B, a) = A • (1 - a) + B • a 

with each color component (colorant) value computed separately. 
10 [001 1 ] Similarly, a prior art system provided compositing without transfer modes but with accumulated alpha as fol- 
lows. 

new_accumulation_alpha = old_accumulation_alpha + element_alpha - old_accumulation_alpha • element_alpha 
new_accumulation_color = InterpolateColor (old_accumulation_color, element_color, element_alpha 
15 /new_accumulation_alpha) 

The foregoing calculation was also used with a transfer mode that was the normal mode, that is, where T(lower, upper) 
= upper. 

[0012] Where compositing involved both a transfer mode and an accumulated alpha, the prior art generally pro- 
20 vided ad hoc solutions designed to work with specific transfer modes or kinds of transfer modes. Such ad hoc solutions 
involved doing things like temporarily matting the results in the accumulation buffer against a solid color or extending 
the accumulation buffer with special values. For example, a prior art technique for dealing with multiply mode is to 
extend the lower color and opacity with white before blending in the upper color. As a consequence, where there is no 
lower color, the result of the blend is the upper color. This technique works if something can play the role of white, but 
25 the technique cannot be applied in general. Another technique of ad hoc prior art systems is simply to expect pixel 
colors to be defined even where the alpha value is zero. In effect, this initializes the accumulation buffer to white or some 
other color wherever alpha is zero. 

[0013] Another ad hoc method does a matte with a color blend, and dematte. This method is applicable where a 
color M exists such that T(M,X) = X, in which case the following calculations are performed. In the calculations, A is the 
30 accumulated_color, a is the accumulated_alpha, B is the element_color, and b is the element_alpha. 

A := M • (1-a) + A • a 
A := A • (1-b) +T(A,B) • b 
a := a + b - a • b 
35 A:=(A-M • (1-a)) /a 

[0014] Yet another ad hoc method extends with element data and then blend. To work properly, this method 
depends on the transfer mode satisfying T(X,X) = X. In terms of the definitions used above, the following calculations 
are performed by this method. 

40 

x := a 

a := a + b - a • b 

t \= x/a 

A:=B • (1 -t) + A • t 
45 A := A • (1 -b)+T(A,B) • b 

[0015] None of the ad hoc methods provides satisfactory results for ColorDodge or ColorBurn transfer modes, and 
none is suitable for use with arbitrary, user-defined transfer modes. 

so SUMMARY OF THE INVENTION 

[0016] The invention features methods and apparatus, including computer program products, operable to blend two 
color-and-alpha pairs using any arbitrary transfer mode function to specify the color blending, providing, in the general 
case, transparency information in the result The invention is particularly useful for accumulating a color-and-alpha value 
55 into an accumulation buffer, but it is generally useful for producing a blended color and alpha value given two color-and- 
alpha pairs and an arbitrary transfer mode. 

[0017] In general, in one aspect, the invention includes blending a lower and an upper graphic element in accord- 
ance with an arbitrary transfer mode by calculating a transfer color from a lower element color and an upper element 
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color according to the transfer mode without regard either to a lower element alpha or to an upper element alpha; cal- 
culating a resulting alpha from the lower element alpha and the upper element alpha without regard to any of the lower 
element color, the upper element color, or the transfer mode; and calculating a resulting color as a weighted avenge of 
the lower element color, the transfer color, and the upper element color. Advantageous implementations can include 

5 one or more of the following features. The weighted average is calculated as an interpolation of the lower element color 
and the upper element color interpolated with the transfer color, where the interpolation of the upper element color is 
governed by the lower element alpha and the interpolation of the lower element color is governed by the upper element 
alpha divided by the resulting alpha. The transfer mode can be user defined. Each of the lower and upper elements can 
have an alpha value between fully transparent and fully opaque, exclusive. Each of the lower and upper elements can 

10 be a pixel or a region. Each of the colors can be defined by a value for each of a plurality of colorants. 

[0018] In general, in another aspect, the invention can be applied to accumulating graphic layers in a layer order. In 
this aspect, the invention includes providing an accumulation buffer having a plurality of graphic elements, each graphic 
element having color and alpha values; compositing the graphic elements of the layers into the accumulation buffer in 
the layer order; for each compositing of an accumulation element and a layer element of a layer having an associated 

15 transfer mode function, (i) calculating a resulting alpha value from the alpha values of the accumulation element and 
the layer element, (ii) calculating a resulting color value from a color value of the accumulation element, a color value 
of the layer element, and the transfer mode function, and (iii) updating the accumulation buffer by storing the resulting 
color value and the resulting alpha value in the accumulation buffer; wherein the resulting color is a weighted avenge of 
the color value of the accumulation element, the color value of the layer element, and the value of the transfer mode 

20 function applied the color value of the accumulation element and the color value of the layer element, the weights for 
the weighted average being determined by the alpha value of the accumulation element and the alpha value of the layer 
element. The invention is advantageously applied in situations where, after accumulating at least one of the layers into 
the accumulation buffer, not all the elements of the accumulation buffer have the same alpha value; where at least one 
of the resulting alpha values stored in the accumulation buffer is an alpha value between fully transparent and fully 

25 opaque, exclusive; where at least one of the elements of at least one of the second through last layers in the layer order 
has an alpha value between fully transparent and fully opaque, exclusive; what the transfer mode associated with at 
least one of the layers lightens the accumulation color, or where the transfer mode associated with at least one of the 
layers applies different lightening or darkening effects depending on the color of the layer element. 
[0019] Advantages that can be seen in implementations of the invention include one or more of the following. The 

30 invention provides fully general handling of transfer modes in compositing with accumulated alpha and does so in a con- 
sistent manner producing sensible, visually continuous and therefore pleasing results, particularly when compositing 
into an accumulation buffer having both zero and non-zero alpha values. The compositing results match the two special 
cases described above — compositing without transfer modes but with accumulated alpha and compositing without 
accumulated alpha but with transfer modes — and do at least as well as current special-case-based solutions. In par- 

35 ticular, for a source color, a source alpha value, and a transfer mode being blended into an accumulation buffer, the fol- 
lowing desirable conditions are satisfied. 

(a) If the accumulation buffer is entirely transparent - i.e, the accumulation alpha value is 0 — there is nothing to 
blend with, and the resulting accumulated color and alpha value is the same as source color and source alpha. 
40 (b) If the accumulation buffer is fully opaque — i.e., the alpha value is 1 — the resulting color is the same as in the 

case where the accumulation buffer simply does not have an alpha component and the accumulated alpha value 
remains 1 . 

(c) The intermediate results are visually continuous — i.e., if the initial accumulation buffer contains an alpha value 
of 1%, then the results are very similar to the case where it contains an alpha value of 0; if the accumulation buffer 

45 contains an alpha value of 99%, then the results are very similar to the 1 00% case. 

(d) If the source alpha value is zero, the resulting accumulated color and alpha value match the initial accumulated 
color and alpha value. 

(e) The intermediate results are visually continuous with changes in the source opacity, (f) Alpha values are com- 
bined in the same way with and without a transfer mode. 

so (g) The compositing operation does not depend on special properties of a selected transfer mode and therefore 

operates easily with new transfer mode functions, including user-defined functions. 

[0020] In addition, the invention simplifies the design of graphics manipulation systems, whether implemented in 
hardware, software, firmware, or any combination of them. A system using the invention can accept as input any 
55 number of layers of graphic elements having any arbitrary selection of color and alpha values, and can further accept 
and associate with any of the layers any arbitrary transfer mode function, and composite the elements according to the 
transfer modes in a fully general way. The invention allows the colors for the accumulation or the source to be arbitrary 
when the accumulated alpha or source alpha is zero, respectively. 
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[0021 ] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the 
description below. Other features and advantages of the invention will become apparent from the description, the draw- 
ings, and the claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0022] 

FIG. 1 illustrates the operation of a system performing a fully general blending operation with transfer mode in 
accordance with the invention. 

FIG. 2 illustrates a computation of the results of the blending operation. 

FIG. 3 illustrates a process of accumulating layers of graphic elements in accordance with the invention. 
[0023] Like reference numbers and designations in the various drawings indicate like elements. 
DETAILED DESCRIPTION 

[0024] As shown in FIG. 1 , in a system or process performing a fully general compositing or blending operation in 
accordance with the invention, a foreground or upper graphic element 102 and a background, lower, or accumulation 
graphic element 104 are blended according to a transfer mode function T(lower, upper) 106. In the general case, each 
graphic element has a color and an alpha, which are shown in the notation (color, alpha). The color itself may include a 
number of elements referred to as colorants. For example, a graphic element in RGB color space includes three color- 
ant values: one each for red, green, and blue. Generally but not invariably, computations performed on colors are per- 
formed on each colorant value separately. 

[0025] The lower and upper color-and-alpha pairs are blended (operation 110) in accordance with the transfer 
mode function 106 to generate a resulting or blended color-and-alpha pair 1 14. 

[0026] The transfer mode function 1 06 can be an arbitrary transfer mode function. The transfer mode can be pre- 
defined in the system or it can be user-defined. A system can receive a user definition of a transfer mode interactively 
or otherwise and in a variety of forms, including as a user definition of a compound transfer mode or as a user definition 
of a transfer mode function T expressed, for example, as a formula or in a plug-in module. An arbitrary transfer mode 
can be used because the fully-general blend operation 110 does not make use of any properties of the transfer mode 
function 1 06. Examples of useful transfer mode functions follow. The normal blend mode produces a weighted average 
of the foreground and background colors as a result of the color interpolation step. The shadow or multiply blend mode 
simulates a shadow by scaling the background color towards dark proportional to the darkness and color of the fore- 
ground color. The glow or screen blend mode simulates a glow by scaling the background color towards light propor- 
tional to the lightness and color of the foreground color. It can also be likened to shining two slide projectors on the same 
screen. The darker blend mode selects the darker of the foreground and background colors. The lighter blend mode 
selects the lighter of the foreground and background colors. The color dodge and color burn blend modes lighten and 
darken colors, roughly simulating the photographic darkroom dodge and burn processes. The soft light blend mode 
applies different lightening or darkening effects depending on the upper color; the effect is similar to shining a diffused 
spotlight on an image. The hue blend mode creates a result color with the luminance and saturation of the lower color 
and the hue of the upper color. The saturation blend mode creates a result color with the luminance and hue of the lower 
color and the saturation of the upper color. The color blend mode creates a result color with the luminance of the lower 
color and the hue and saturation of the upper color. The luminosity blend mode creates a result color with the hue and 
saturation of the lower color and the luminance of the upper color. The add blend mode simulates two combined light 
sources by adding color intensities. The overprint blend mode simulates overprinting of two inks by adding ink color val- 
ues. Use of the general blend operation 1 10 is particularly advantageous with the color dodge and soft light transfer 
modes. 

[0027] As shown in FIG. 2, and in the terminology of layer accumulation, the following formulas define the fully gen- 
eral blending operation 1 1 0 (BlendWithTransferMode) with transfer mode function T. 
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new_accumulation_alpha 

- InterpolateAlpha (old_accuraulation_alpha, 1# element^alpha) 
5 new_accumulation_color » 

InterpolateColor {old_accumulation_ color/ 
InterpolateColor (element_color, 

T (old_accumulation_color, element_color) # 

10 



old accumulation alpha) , 

15 — — 

element alpha / new accumulation alpha) 



20 [0028] In terms of two color-and-alpha pairs, <A,a) and <B,b), which are the lower and upper colors, respectively, a 
transfer mode function T, and conventional linear interpolation for alpha and color values, the definition of fully general 
blending can be expressed as follows. The result of the blending operation is <C,c). The parts of the result are defined 
by the following formulas. 

25 c = InterpolateAlpha (a, 1 , b) 
t = b/c 

C = InterpolateColor (A, InterpolateColor (B, T (A, B), a), t) 
With linear interpolation for alpha and color values, the formula reduce to the following. 

30 

c = a + b - a • b 
t = b/c 

C = A • (1 - 1) + B • (1 - a) • t + T(A, B) • a • t 

35 The new color formula uses more variables than were needed to perform the prior art calculations described earlier. As 
a result, more temporary resources are needed to keep values for processing. 

[0029] Assuming that the element components <B, b) are not writable and that one wants to process the accumula- 
tion component <A,a) in place, the computation can be done as follows: 

40 blended_color := T (accumulation_color, element_color) 

blended_color := InterpolateColor (element_color, blended_color, accumulation_alpha) 

accumulation_alpha := InterpolateAlpha (accumulation_alpha, 1, element_alpha) 

accumulation_color := InterpolateColor (accumulation_color, blended_color, element_alpha / accumulation_alpha) 
45 or 

X:=T(A, B) 

X := InterpolateColor (B, X, a) 
a := InterpolateAlpha (a, 1 , b) 
so A := InterpolateColor (A, X, b/a) 

In the case of linear interpolation for alpha and color values, with optional use of a temporary variable k, this becomes: 

X := T (A, B) 

55 X := B • (1 - a) + X • a (or, equivalent^, X := B + (X - B) • a ) 

a := a + b - a • b 
k := b/a 

A := A • (1 - k) + X • k (or, equivalent^, A := A + (X - A) • k ) 
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[0030] As has been noted, the interpolation provided by InterpolateAlpha and InterpolateColor need not be a linear 
interpolation. For example, InterpolateColor can be defined to do different things in different color spaces, subject to the 
usability constraints that it have appropriate endpoint behaviors and appear continuous and pleasing in between. As an 
example, one might want to express the colors in one color space but do the actual interpolation in another color space 
(e.g., interpolating in the CIE XYZ color space). As another example, one might want to convert gamma-encoded val- 
ues to unencoded values, interpolate the unencoded values, and then convert the result back as the result of the inter- 
polation. 

[0031 ] FIG. 3 illustrates the use of the fully general blending operation in compositing a stack of two or more blend 
layers to build a final image. In practice, each layer will typically include both image data and compositing controls. The 
compositing controls may include a transfer mode, also known as a blending mode, which is represented for computa- 
tion by a transfer mode function T, as has been described. The image data for a layer is represented by graphic ele- 
ments, such as pixels arranged in a grid. 

[0032] A stack of layers is composited by initializing an accumulation buffer and then compositing layer elements 
from bottom to top into the accumulation buffer. The process obtains the values of the elements of the accumulation 
buffer and of the current layer (steps 302, 310). It may be necessary to calculate the graphic elements of the current 
layer. Then, the color of each element in the layer is composited with the color of the corresponding element in the accu- 
mulation buffer to generate a new composited color (step 312). This is done using the BlendWithTransferMode opera- 
tion 110 (FIG. 1); the combination is controlled by the opacities of the two elements and the transfer mode, and the 
elements in the accumulation buffer have calculated (that is, not implicit) alpha values. After all the elements of the cur- 
rent, bottom-most layer have been composited ('no' branch from decision step 320), the process continues with the next 
layer ('yes' branch from decision step 322) until all elements and layers have been processed bottom to top. The final 
contents of the accumulation buffer represent the final image. 

[0033] The foregoing formulas and processes may perhaps be better understood in light of the following analysis. 
[0034] The known crossfade operation 

<C,c) = XFade ( <A,a), <B,b), u ) 
is defined as follows. When u is 0, 

<C,c) = <A,a>. 
When u is 1 , 

<C,c) = <B,b). 

Intermediate values were calculated as follows: 

c = a • (1 - u) + b • u 

t = b • u / c 

C = A • (1 - 1) + B • t 

This definition is now generalized as follows. 

c = InterpolateAlpha (a, b, u) 
t = b • u / c 

C = InterpolateColor (A, B, t) 

[0035] Crossfade is the operation to use when blending between an accumulation buffer representing an image 
before applying masked elements and the accumulation buffer representing the image after applying the masked ele- 
ments. 

[0036] Now consider the blending function Blend ( <A,a), <B,b) ), which composites upper element <B,b) into lower 
(accumulation) element <A,a). The familiar linear compositing equations teach that the result <C,c) is defined by: 

c = a + b - a • b 
t = b/c 

C = A • (1 - 1) + B • t 

It is straightforward to confirm that applying a mask u directly to (B,b) to produce (B, b • u) and compositing this into (A,a) 
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produces the same results as using the blend and crossfade approach. That is: 
Blend «A,a>, <B,b • u» = XFade ( <A,a>, Blend «A,a), <B,b», u ) 

5 This permits one to represent compositing <B,b) into <A,a) as treating all of the alpha component for <B,b) as part of the 
masking process. That is: 

Blend ( <A,a), <B,b) ) = XFade ( <A,a), Blend ( <A,a), <B,1) ), b ) 

w This follows from the foregoing equations. 
Furthermore: 

Blend ( <A,a>, <B,b> ) = XFade ( <B,b>, Blend «A,1>, <B,b> ), a ). 

15 Thus, Blend «A,a), <B,b» can be computed as follows. 

Blend «A,a>, <B,b» 

= XFade «A,a>, Blend ( <A,a), <B,1> ), b ) 

= XFade «A,a>, XFade ( <B,1>, Blend ( <A,1>, <B,1> ), a ), b ) 



20 



25 



Into this formula one can introduce the transfer mode function T(lower, upper), as follows: 

BlendWithTransferMode «A,a), <B,b), T) 

= XFade ( (A,a), XFade ( <B,1>, <T(A,B), 1), a ), b ) 

The middle term can be reduced as follows. 



XFade ( (B,1), (T(A,B), 1), a ) = <X, x> 
x = InterpolateAlpha (1 , 1 , a) 
30 = 1 

s = 1 • a / x 
= a 

X = InterpolateColor (B, T(A,B), s) 

= InterpolateColor (B, T(A,B), a) 

35 

From this it can be seen that, where the interpolations are linear, the result 

<C,c> = BlendWithTransferMode «A,a), <B,b), T) 

40 is defined by: 

c = a + b - a • b 
t + b/c 

C = A • (1 - 1) + B • (1 - a) • t + T(A,B) • a • t 



45 



[0037] Without the condition that the interpolations be linear, the result <C,c) is defined by: 



C = InterpolateAlpha (a, 1 , b) 
t = b/c 

so C = InterpolateColor (A, InterpolateColor (B, T(A,B), a), t) 

[0038] InterpolateColor can be generalized to be a weighted averaging operation for colors - i.e., a function that 
takes n colors and n weights between 0 and 1 that sum to 1 and generates a resulting color. For this condition to be 
satisfied requires, for example, that InterpolateColor satisfy the following equation: 



55 



InterpolateColor (A, InterpolateColor (A, B, u), v) 
= InterpolateColor (A, B, u • v) 



9 



EP 1 050 852 A2 



As a weighted averaging operation for colors, InterpolateColor, as applied above, averages the colors A, B, and T(A,B) 
using the weights (1 -t), (1 -a) • t, and a • t. Expanding out the weights, one finds that their respective values are: 

a • (1 - b) / c 
5 (1 - a) • b/c 

a • b/c. 

[0039] The compositing model that has been used in the described implementation is essentially the Porter-Duff 
"over" operator. However, the principles of the invention are applicable to other compositing models. For example, one 
10 can use the "atop" operator to define the compositing model, sometimes referred to as clipped blending, which basically 
blends as if the underlying element had an alpha value of 1 and then using the underlying element's alpha value in the 
result. With clipped blending as the compositing model, it turns out that the result <C,c) is defined by: 

c = a 

15 C= InterpolateColor (A, T(A, B), b) 

[0040] The result derived above, namely 

C = InterpolateColor (A, InterpolateColor (B, T(A,B), a), t) 

20 

has a dual, which can be seen by recognizing the identity that 

A OVER B = (A ATOP B) OVER A. 

25 That is, the following dual result can also be used. 

<X,x) = <B,b) ATOP <A,a) with transfer mode T 
<C,c) = <X,x> OVER{B,b) 

30 While the calculations presented earlier preserve the blended color across the alpha calculation, with this dual result 
one need only preserve the old accumulated alpha across the new accumulated alpha calculation. The resulting steps, 
then, are as follows: 

blended_color : = T (accumulation_color, element_color) 

35 accumulation_color := InterpolateColor (accumulation_color, blended_color), element_alpha) 
old_accumulation_alpha := accumulation_alpha 

accumulation_alpha := InterpolateAlpha (accumulation_alpha, 1, element_alpha) 

accumulation_color := InterpolateColor (B, accumulation_color, old_accumulation_alpha / accumulation_alpha) 

40 Thus, the parts of the result <C,c) = BlendWithTransferMode «A,a), <B,b),T) can also be defined by the following formulas. 

c = InterpolateAlpha (a, 1 , b) 
t = a/c 

C = InterpolateColor (B, InterpolateColor (A, T (A, B), b), t) 

45 

With linear interpolation of alpha and definitions of InterpolateColor based on weighted color averaging, this expands 
to the same formula given earlier for 

C = InterpolateColor (A, InterpolateColor (B, T(A,B), a), t) 

50 

with 

t = b/c 
55 namely 

C = A • a • (1 - b)/c + B • (1 - a) • b/c + T(A,B) • a • b/c 
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[0041] The computation of the dual form may be simpler in the case of linear interpolations, as shown in the follow- 
ing steps, with an optional temporary variable t. 

A:= A*(1-b)+T(A,B)'b 
5 X := a 

a := a + b - a • b 
t := x/a 

A := B«(1-t) + A«t 

w [0042] The invention has been described in terms of particular embodiments. Other embodiments are within the 
scope of the following claims. For example, the steps of the invention can be performed in a different order and still 
achieve desirable results. Thus, although the process of compositing has been described as proceeding layer-by-layer, 
the compositing can proceed in other orders, as long as the compositing for each element position occurs in the proper 
order. The invention can be implemented in a vector context as well as in the described raster context. The invention 

15 can be implemented for graphics elements that are regions, such as PostScript paths. 

[0043] The invention has generally been described in terms of loops of the form "for each layer do {for each pixel 
do {} }". The operations can also be organized in loops of the form "for each pixel do {for each layer do {} }", in loops of 
the form "for each region do { for each layer do {for each pixel in the region do {}}}", in loops of the form "for each region 
do { for each pixel in the region do {for each layer do {}}}", and so on. In performing the operations of the invention ana- 

20 lytically, one could find all of the primitive regions and then compute their colors, or accumulate a changing set of 
regions complete with colors as each layer is added. 

Claims 

25 1. A method of blending a lower graphic element and an upper graphic element in accordance with a transfer mode 
associated with to upper graphic element, each element having a color and an alpha, comprising: 

calculating a transfer color from the lower element color and the upper element color according to the transfer 
mode without regard either to the lower element alpha or to the upper element alpha; 
30 calculating a resulting alpha from the lower element alpha and the upper element alpha without regard to any 

of the lower element color, the upper element color, or the transfer mode; and 

calculating a resulting color as a weighted avenge of the lower element color, the transfer color, and the upper 
element color. 

35 2. The method of claim 1 , wherein the weighted average is calculated as an interpolation of the lower element color 
and the upper element color interpolated with the transfer color, where the interpolation of the upper element color 
is governed by the lower element alpha and the interpolation of the lower element color is governed by the upper 
element alpha divided by the resulting alpha. 

40 3. The method of claim 1 , wherein: 

the lower element has a color value A and an alpha value a, the upper element has a color value B and alpha 
value b, and the transfer mode is defined by transfer mode function T; 
to transfer color is calculated as T(A,B); 
45 the resulting alpha c is calculated from the values a, b and 1 ; and 

the resulting color C is calculated as a weighted average of the colors A, B, and T(A,B), with weights deter- 
mined by the values a and b. 

4. The method of claim 3, further comprising: 

50 

receiving from a user a definition of the transfer mode. 

5. The method of claim 3, wherein the weights for the weighted average of the colors A, the colors B, and the transfer 
colors T(A,B) are (1 - (b / c)), (1 - a) • b / c, and a • b / c, respectively. 

55 

6. The method of claim 5, wherein the calculated values are: 

c = a + b - a • b 
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C = A • (1 - (b / c)) + B • (1 - a) • (b / c) + T(A, B) • a • (b / c). 

7. The method of claim 3, wherein the resulting graphic element <C,c) having color value C and alpha value c is calcu- 
lated in accordance with 

(C,c) = XFade ( (A, a), XFade ( <B,1>, <T(A,B), D, a ), b ), 

where XFade «X,x), <Y,y>, z) = <R,r) is an operation defined by the formulas 

r = InterpolateAlpha (x, y, z), 

t = y »z/ r, and 

R = InterpolateColor (X, Y, t). 

8. The method of claim 7, wherein: 

each of the lower and upper elements has an alpha value between fully transparent and fully opaque, exclu- 
sive. 

9. The method of claim 7, wherein: 

each of the lower and upper elements is a pixel. 

10. The method of claim 7, wherein: 

InterpolateAlpha is a linear interpolation function yielding r = x • (1 - z) + y • z; and 
InterpolateColor is a liner interpolation function yielding R = X • (1 - 1) + Y • t. 

1 1 . The method of claim 1 0, wherein: 

each of the lower and upper elements has an alpha value between fully transparent and fully opaque, exclu- 
sive. 

12. The method of claim 1 0, wherein: 

each of the lower and upper elements is a pixel. 

13. The method of claim 7, wherein: 

each of the lower and upper elements is a region. 

14. The method of claim 7, wherein: 

each of the colors A, B, and C is defined by a value for each of a plurality of colorants. 

15. The method of claim 7, further comprising: 

receiving from a user a definition of the transfer mode. 

16. A method of compositing a lower graphic element (A, a) and an upper graphic element (B,b) with a transfer mode 
defined by a transfer mode function T, the lower element having a color value A and an alpha value a, the upper 
element having a color value B and an alpha value b, the method comprising: 

calculating a resulting graphic clement <C,c) having color value C and alpha value c in accordance with 

(C,c> = XFade ( <A,a>, XFade ( <B,1), <T(A,B), 1), a ), b ), 

where XFade «X,x), <Y,y), z) = <R,r) is an operation defined by the formulas 
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r = InterpolateAlpha (x, y, z), 
t = y • z / r, and 

R = InterpolateColor (X, Y, t); and 
5 where the alpha value a is not equal to 1 and the function T(A ,B) is different from B for some value of A. 

17. The method of claim 1 6, wherein: 

each of the upper and lower elements is a pixel having an alpha value between fully transparent and fully 
w opaque, exclusive. 

18. The method of claim 1 7, wherein: 

InterpolateAlpha is a linear interpolation function yielding r = x • (1 -z) + y • z; and 
15 InterpolateColor is a linear interpolation function yielding R = X • (1 -t) + Y • t. 

each of the graphic elements is a region. 

19. A method of accumulating graphic layers in a layer order, each layer comprising a plurality of graphic elements, 
comprising: 

20 

providing an accumulation buffer having a plurality of graphic elements, each graphic element having color and 
alpha values; 

compositing the graphic elements of the layers into the accumulation buffer in the layer order; 

for each compositing of an accumulation element and a layer element of a layer having an associated transfer 

25 mode function, (i) calculating a resulting alpha value from the alpha values of the accumulation element and 

the layer element, (ii) calculating a resulting color value from a color value of the accumulation element, a color 
value of the layer element, and the transfer mode function, and (iii) updating the accumulation buffer by storing 
the resulting color value and the resulting alpha value in the accumulation buffer; wherein 
the resulting color is a weighted average of the color value of the accumulation element, the color value of the 

30 layer element, and the value of the transfer mode function applied to the color value of the accumulation ele- 

ment and the color value of the layer element, the weights for the weighted average being determined by the 
alpha value of the accumulation element and the alpha value of the layer element. 

20. The method of claim 19, wherein, after accumulating at least one of the layers into the accumulation buffer, not all 
35 the elements of the accumulation buffer have the same alpha value. 

21. The method of claim 19, wherein at least one of the resulting alpha values stored in the accumulation buffer is an 
alpha value between fully transparent and fully opaque, exclusive. 

40 22. The method of claim 21 , wherein at least one of the elements of at least one of the second through last layers in 
the layer order has an alpha value between fully transparent and fully opaque, exclusive. 

23. The method of claim 19, wherein, after accumulating at least one of the layers into the accumulation buffer, not all 
the elements of the accumulation buffer have the same alpha value. 

45 

24. The method of claim 19, wherein at least one of the resulting alpha values stored in the accumulation buffer is an 
alpha value between fully transparent and fully opaque, exclusive. 

25. The method of claim 1 9, wherein the transfer mode associated with at least one of the layers lightens the accumu- 
50 lation color. 

26. The method of claim 25, wherein the transfer mode is represented by a function T that calculates a blended color 
from a lower color L and an upper color U in accordance with T(L, U) = min(1 , L/(1 -U)), with 0/0 treated as 0. 

55 27. The method of claim 1 9, wherein the transfer mode associated with at least one of the layers applies different light- 
ening or darkening effects depending on the color of the layer element. 

28. The method of claim 27, wherein the transfer mode is represented by a function T that calculates a blended color 
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from a lower color L and an upper color U in accordance with 

T(L,U) = L - (1 - 2 • U) • L • (1 - L) if U < 0.5 and 
T(L, U) = L + (2 • U - 1 ) • (D(L) - L) if U > 0.5. 

5 

where 

D(x) = ((16-x- 12)*x + 4)-xif x<0.25, and 
D(x) = sqrt (x) if x > 0.25. 

10 

29. The method of claim 1 9, wherein the transfer mode is user defined. 

30. A system for blending a lower graphic element and an upper graphic element in accordance with an arbitrary trans- 
fer mode associated with the upper graphic element, each element having a color and an alpha, comprising: 

15 

means for calculating a transfer color from the lower element color and the upper element color according to 
the transfer mode without regard either to the lower element alpha or to the upper element alpha; 
means for calculating a resulting alpha from the lower element alpha and the upper element alpha without 
regard to any of the lower element color, the upper element color, or the transfer mode; and 
20 means for calculating a resulting color as a weighted average of the lower element color, the transfer color, and 

the upper element color. 

31. The system of claim 30, wherein the weighted average is calculated as an interpolation of the lower element color 
and the upper element color interpolated with the transfer color, where the interpolation of the upper element color 

25 is governed by the lower element alpha and the interpolation of the lower element color is governed by the upper 
element alpha divided by the resulting alpha. 

32. The system of claim 30, wherein: 

30 the lower element has a color value A and an alpha value a, the upper element has a color value B and an 

alpha value b, and the transfer mode is defined by transfer mode function T; 
the transfer color is calculated as T(A,B); 

the resulting alpha c is calculated from the values a, b and 1 ; and 

the resulting color C is calculated as a weighted average of the colon A, B, and T(A,B), with weights determined 
35 by the values a and b. 

33. The system of claim 32, further comprising: 

means for receiving from a user a definition of the transfer mode. 

40 

34. The system of claim 32, wherein the calculated values are: 

c = a + b - a • b 

C = A • (1 - (b / c)) + B • (1 - a) • (b / c) + T(A, B) • a • (b / c). 

45 

35. The system of claim 32, wherein the resulting graphic element <C,c) having color value C and alpha value c is cal- 
culated in accordance with 

(C,c) = XFade ( (A, a), XFade ( <B,1), <T(A,B), 1), a ), b ), 

50 

where XFade «X,x), <Y,y), z) = (R,r) is an operation defined by the formulas 

r = InterpolateAlpha (x, y, z), 
t = y *z/ r, and 
55 R = InterpolateColor (X, Y, t). 

36. The system of claim 32, wherein: 
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each of the colors A, B, and C is defined by a value for each of a plurality of colorants. 

37. A computer program product, tangibly stored on a computer-readable medium, for blending a lower graphic ele- 
ment and an upper graphic element in accordance with a transfer mode associated with the upper graphic element, 

5 each element having a color and an alpha, the product comprising instructions operable to cause a computer to: 

calculate a transfer color from the lower element color and the upper element color according to the transfer 
mode without regard either to the lower element alpha or to the upper element alpha; 

calculate a resulting alpha from the lower element alpha and the upper element alpha without regard to any of 
10 the lower element color, the upper element color, or the transfer mode; and 

calculate a resulting color as a weighted average of the lower element color, the transfer color, and the upper 
element color. 

38. The product of claim 37, wherein: 

15 

each of the lower and upper elements has an alpha value between fully transparent and fully opaque, exclu- 
sive. 

39. The product of claim 37, wherein: 

20 

each of the lower and upper elements is a pixel. 

40. The product of claim 37, wherein: 

25 each of the lower and upper elements is a region. 

41. The product of claim 37, wherein: 

each of the lower graphic element color, the upper graphic element color, and the resulting color is defined by 
30 a value for each of a plurality of colorants. 

42. The product of claim 37, wherein the weighted average is calculated as an interpolation of the lower element color 
and the upper element color interpolated with the transfer color, where the interpolation of the upper element color 
is governed by the lower element alpha and the interpolation of the lower element color is governed by the upper 

35 element alpha divided by the resulting alpha. 

43. The product of claim 37, wherein: 

the lower element has a color value A and an alpha value a, the upper element has a color value B and an 
40 alpha value b, and the transfer mode is defined by transfer mode function T; 

the transfer color is calculated as T(A,B); 

the resulting alpha c is calculated from the values a, b and 1; and 

the resulting color C is calculated as a weighted average of the colors A, B, and T(A,B), with weights deter- 
mined by the values a and b. 

45 

44. The product of claim 43, further comprising instructions to: 

receive from a user a definition of the transfer mode. 

so 45. The product of claim 44, wherein the definition of the transfer mode is received in the form of a definition of a com- 
pound transfer mode. 

46. The product of claim 44, wherein the definition of the transfer mode is received in the form of a formula defining the 
function T. 

55 

47. The product of claim 43, wherein the weights for the weighted average of the colors A, B, and T(A,B) are (1 - (b / 
c)), (1 - a) • b / c, and a • b / c, respectively. 
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48. The product of claim 47, wherein the calculated values are: 

c = a + b - a • b 

C = A • (1 - (b / c)) + B • (1 - a) • (b / c) + T(A, B) • a • (b / c). 

49. The product of claim 43, wherein the resulting graphic element (C,c) having a color value C and an alpha value c is 
calculated in accordance with 

<C,c> = XFade ( <A,a>, XFade ( <B,1), <T(A,B), 1),a ), b ), 

where XFade «X,x), <Y,y), z) = (R,r> is an operation defined by the formulas 

r = InterpolateAlpha (x, y, z), 

t = y »z / r, and 

R = InterpolateColor (X, Y, t). 

50. The product of claim 49, wherein: 

InterpolateAlpha is a linear interpolation function yielding r = x • (1 - z) + y • z; and 
InterpolateColor is a linear interpolation function yielding R = X • (1 - 1) + Y *t. 

51. The product of claim 50, wherein: 

each of the lower and upper elements has an alpha value between fully transparent and fully opaque, exclu- 
sive. 

52. The product of claim 50, wherein: 

each of the lower and upper elements is a pixel. 

53. The product of claim 37 for accumulating graphic layers in a layer order, each layer comprising a plurality of upper 
graphic elements, the product further comprising instructions to: 

provide an accumulation buffer having a plurality of graphic elements, each graphic element having color and 
alpha values; 

composite the graphic elements of the layers into the accumulation buffer in the layer order; and 

for each compositing of an accumulation element and a layer element of a layer having an associated transfer 

mode function, calculate the resulting alpha and the resulting color as recited in claim 37. 
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transfer mode by calculating a transfer color from 
a lower element color and an upper element color 
according to the transfer mode without regard 
either to a lower element alpha or to an upper 
element alpha; calculating a resulting alpha from 
the lower element alpha and the upper element 
alpha without regard to any of the lower element 
color, the upper element color, or the transfer 
mode; and calculating a resulting color as a 
weighted average of the lower element color, the 
transfer color, and the upper element color. The 
transfer mode can be an arbitrary transfer mode 
and can be user defined . The resulting graphic 
element &lang&C, c&rang& having color value C and 
alpha value c can be calculated in accordance with 
&lang&C, c&rang& = XFade ( &lang&A, a&rang& , XFade 
( &lang&B, l&rang&, &lang&T (A, B) , l&rang&, a), b) , 
where XFade ( &lang&X, x&rang&, &lang&Y, y&rang& , z) 
= &lang&R, r &rang& is an operation defined by the 
formulas r = InterpolateAlpha (x, y, z), t = y.z / 
r, and R = InterpolateColor (X, Y, t) and where 
the lower element is &lang&A, a&rang& and the upper 
element is &lang&B, b&rang& . Each of the colon A, 
B, and C can be defined by a value for each of a 
plurality of colorants. The calculated values can 
defined by c = a + b - a.b, and C = A. (1 - (b / 
c)) + B.(l - 1) . (b / c) + T (A, B).a.(b / c) . 
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